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ABSTRACT 

The 1978 edition of the Metropolitan Achievement 
Tests was analysed for sex stereotyping and for the use o'f male, 
female, or neuter references and a cossparison was made with the 1970 
edition. There was less bias in the new edition, and there was 
relatively little, sex stereotyping with respect to occupations, 
activities, and roles for females. There was, however some sex 
stereotyping with respect to the display of emotions. The index of 
sex bi-as used was the ratio of male nouns and pronouns to female, 
with a correction for repeated references to the saoTe person. A 
similar index compute^ the ratio cf neutral personal nouns and 
pronouns tfi**gender-specif ic words. Illustrations were coded similarly 
for gender ratios and also analyzed for the comparative stature of 
malets and females. The possible effects of sex bias and seX 
stereotyping in test content on test performance are yet to be' 
determined; test developers have assumed that an even balance of male 
and female references would ainimize test bias. The 1978 edition of 
the MAT is more, balanced than the 1970 edition, and generally 
presents a less stereotyped portrayal 6f both sexes. (CTM) 
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Recent years have brought increased attention to gender balance in evalua- 
tion .devices . Research indicates that current educational materials present 
males significantly^more often than females, and both sexes almost exclu- 
sively in stereotypic roles. In revising the Metropolitan Achievement ' 
Tests , the publisher attempted to balance the presentation of sexes and to 
portray both sexes in less stereotypic roles. This study was undertaken to 
determine whether, the revision conformed to these guidelines. Results in- 
dicate the test content is significantly better ^balanced ancj Jess stereotypic 
than previous tests. Breakdowns of . test content and various thei4tics' 
categories are presented; data are compared across content areas and test 
'levels. 
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GENDER BALANCE ANALYSIS OF THE METROPOLITAN ACHIEVEMENT TESTS , 1978 EDITION 
f 

Beginning with«a NOW* publ ication regarding the sex-role stereotyping in publi.c 
school reading texts (Committee on Women in Words and Images, 1972), there has 
been a growing body of literature concerning such stereotyping 1n all curriculOm 
materials, including standardized tests. Several studies and position papers 
have been published in connection with the issue (c.f., Beck, 1975; Educational 
Testing Service,' 1976; Faggen-Steckl er , McCarthy, & Tittle, 1974; Saario, 
Jacklin, & Tittle, 1973). 

/ 

Faggen-Steckler, et a] . (197*) looked specifically at the item content of eight 
standardized achievement tests. Their purpose was to examine "sex imbalance as 
well as... stereotyped representations of women." Their measure of sex imbalance 
was the ratio of male nouns and pronouns to .female nouns and Dronouns. A 
balanced test woul-d-have a ratio of approximately 1.0; female nouns and pronouns 
should be usedj'n items as often 'as male nouns and pronouns. The results showed 
ratios ranging from .86 (100 female references foAavery 86 male references) to 
14.0 (14 male references for each female reference) N 

In his guidelines for reducing "sex bias," Beck ( 1 975) Suggested that males and 
females should be represented with equal frequency wi thi rl a full range of 
activities and vocations. Moreover, attempts shbuld be made to maximize the use 
of neuter references in standardized tests. This could be accomplished by use of 
neuter titles or generic descri ptions* or by re-wording gender-specif singular 
phrases to form neutral, plural phrases. 
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Educational Testing Service\l976) and the McGraw-Hill Book Company (1974), 
suggest similar guidelines for, their publications. ETS adds the warning that 
re-phrasing, should not result in wording that is complex or confusing. 



Current commercial achievement'teSts are not sufficiently balanced in regard 
to sex-role references (Faggen-St^ckler , et al^_1974). Therefore, as new 
tests are published and existing tests are revised, there will probably be 
increasing attention paid to sexist termi nol ogy^and overtly biased attitudes. 
The present analysis was undertaken to determine the exteqt to which the 1978 
edition of the Metropolitan Achievement Tests (MAT) conforms to certain 
guidelines for attaining a balanced representation of sexes and sex roles. 
The test was examined with respect 'to its overall gender-balance (i.e., use of 
gender-specific nouns and pronouns), as well as its portrayal of males and 
females ifs^diverse activities. 

In the final version, the 1978 MAT consists of two forms (J and K) of^each of 
four components: Instructional Tests in Mathematics, Reading, and Language 
and a Survey Battery. The Instructional Tests are available at six levels 
(Primer to Advanced 1) and the Survey is available at eight levels (Preprimer to 
Advanced 2). The present investigation includes only Form J of all components 
and levels. 1 

Procedure 

All test booklets (text and illustrations) of the 1978 MAT were analyzed. The 
test content was examined using both a quantitative and a qualitative method. 



] The authors are grateful to T he Psychological Corporation for allowing them 
atcess to the pre-publication materials'. 
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The quantitive method was a variation of the one used by Faggen-Steckl er, et al . , 
(1974). A tally of male nouns and propduns femal e nouns and pronouns, and 

neuter nouns and pronouns was made across a]l components and levels. 

•'I i 

Two ratios were calculated: if. / 

1 ) The Gender Ratio (G^S* „• 

Male Nouns (MN) -h/Male Pronouns (MP ) 
Female Nouns (FN'tf + F J ema>e Pronouns (FP) 

2) The Neuter Ratio, f!$R) 



Neuter Nouns (M) + Neuter Pronouns (NP ) 
(MN + W) ■ + (FN + FP) ! , 



The MAT authors attempted to resjMce use of gender-specific terms in favor of gender 
neutral terms such as "chil dre'rjj*, "people", "they", "we", etc. It was thought 
'that the second ratio might reflect the degree of success in this effort to 

f 

neutralize the language. It * 

J 

The qualitative analysis involved coding the test items according to four bro^ 
categories: active or pa^ive roles , and traditional and non-traditional occupa- 
tions , activities , and e motions (See Appendix*/*). Items were either coded 
in one or more category/or judged to be neutral. Those judged neutral were 
assigned to none of tffe preceding categories. 

The following guidelines were used for coding the written material: 

1) A noun^or pronoun was considered to be neuter when it was used i_n 
place of^a female or male reference.. For e'xample, "people" (instead of 
"men")'was considered a neuter noun; however references suctr as "the 
•dog/, .it" were not considered neuter, but were omitted from the counts. 



•t 



/ 



2) Each noun or pronoun was counted the first time it occured in a 
passage or an item. Repeated occurrences within the same passage or in 
items referring to the passage were not_ counted. 

3) The thematic codes (occupations, activities, roles, and emotions) 
were applied only to the correct responses for each item. Nouns and » 
pronouns were counted in correct and distractor responses. 

4) Boys and girls engaged in school-related activities were judged to 
be in male or female traditional activities. 

5) References to "mother," "father," "grandmother," or "grandfather" 
were coded as occurrences of traditional activities for each gender. 

♦ 

All illustrations were coded along the same thematic dimensions. Frequency 
counts of males and females were made and categorized with respect to tradi- 
tional or non-traditional clothing, and their physical stature (e.g., male and 
female figures the same height, *male or female taller). See Appendix A. 
A tally was also kept of children and adults in mixed-sex or same-sex groups. 

The foil owing .guidelines were applied to the coding of illustrations: 

1) Only the correct responses were analyzed along the thematic dimension. 
All responses were included in the frequency counts of males and females, 
^types of clothing, physical stature and group composition. 

Physical stature was analyzed within age groups: adults compared with 
' adilts, children with children. However, if it was clear one child^/as 
drawn larger than ..another to indicate a difference in age, this was not • 
coded' as a difference in physical stature. 

- 4 - 
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Results and Discussion 

Table 1 shows the Gender Ratio (GR) for each level within each component of 
the 1978 MAT. The GR f s range from .50 to 2.04 with an overall median GR of 1.02. 
The Instructional Mathematics Tests, with a median GR of '.86, displays the greatest 
disparity from the optimal GR of 1.00; and the Survey Batteries (median GR of 1.11) 
show the greatest disproportion of males. As a whole, however, Form J of the MAT 
seems to have a good gender balance. 

Table 2, GR's from the 1978 MAT Survey Batteries are compared with the dat^ 

presented by Faggen-Steckl er , et al., / (1974) for the 1970 edition of MAT. Faggen- 

Steckler, et al. computed a ratio of male "regular" and "generic" nouns and 

pronouns to female "regular" and "generic" nouns and pronouns. They defined 

generic nouns and pronouns as those which "refer to males or females or to a 

combined group" and .for which "there is only one way for referencing the noun in 

question" (p. 153). The present analysis classified the generic terms as either 

male, female, or neuter based upon their pronoun reference* It appears that both 

methods would give essentially the same result with respect to whether a noun or 

pronoun was male or female. Therefore, 1t was concluded, that the GR's for the 

1978 MAT could be compared with the data of Faggen-Steckler, et al . (1974) for the 

1970 MAT. Data for comparable 1970 and 1978 levels only are shown. 

■ * ♦ • 

The- 1970 MAT had a range of ratios from .86 to 4.11 with a median of 2.95. Clearly, 

'the 1978 MAT survey, with a range of .87 to 1.91 and a median of 1.21 shows marked 

improvement over the earlier edition. 
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TABLE 1 



Gender Ratios (6R) of Male Nouns & Pronouns to Female Nouns & Pronouns 

by MAT Component and Level 

MN + MP ' Me.dian GR by level 

Instructional Test FN + FP : GR for Component 

Mathematics « 

Primer Level 13/16 .81 $> 

Primary 1 17/21 .81 • 

Primary 2 21/24 .88 .81 

Elementary 26/20 1.30 

Intermediate * 9/15 .60 

Advanced 1 6/12 -50 

Total: Mathematics 92/108 .85 

/ 

Reading , 

Primer 28/16 ' • . 1.75 

Primary 1 ' 56/59 ,95 

.Primary 2 46/49 >94 

Elementary , 47/44 1.07 

Intermediate -45/43 • ^ 1.05 

Advanced 1. 67/33 2J)3 

Total: Reading 289/244 ' 1.18 



46/49 )94 < 1-06 7 



Language 

Primer . " 30/38 • .79 

Primary 1 53/36 1.47 

Primary 2 64/62 1.03 1.06 

Elementary c 75/74 1.01 

Intermediate 55/51 1.08 

Advanced 1 . " 50/32 1J3£ 

Total: Language 327/293 1.12 



Survey Batteries 

Preprimer • 22/33 .67 

Primer ' 31/31 1.00 . 

Primary 1 74/61 1.21 

Primary 2 ' ' 66/76 .87 1.11 

Elementary ' 69/68 T _ 1.01 

Intermediate • 63/33 x 1.91 

Advanced 1 64/51 1.25 

Advanced 2 47/23 

Total: Survey -436/376 1.16 



In both ed-itions of th'e MAT, there appears to be a trepd-toward higher GR's 
at the upper levels of the test. The J|Jfence*and Social Studies Tests in those 
upper levels include biographical material about famous people in those fields. 
v As these achievement tests reflect &irrent curriculum materials, which to a ( 
large extent still emphasize the contributions of men in these areas, it is 
inevitable that the tests contain an excess .of male references. 
This may partially account fofjhe' higher proportion of male references in the 
upper levels of the complete Survey Batteries. (* 

( . , TABLE 2 

Gender-Ratios (GR) by Level for the 1970 MAT* & 1978 MAT Survey Tests 

- 1970 Edition 



MN + MP 

Test Level FN + FP 1 GR 

Primary 1 51/59 ^ .86 

Primary 2 _ 137/86 Y 1.59 

Elementary 124/42 / . 2.95 

Intermediate 181/44 7 4.11 

Advanced 198/51 3.88 

Median 2.95 

Combined Across Levels 691/28^ 2.45 



1-978 


Edi tion 


MN + MP 




FN + FP 


GR 


74/61 


1.21 


66/76 


.87 


69/68 


1 .01 


* 63/33 


1.91 


64/51 


1 . 25** 




1*1 


336/289 


1 .16 




Table 3 shows the combined GR's\at each level for the 1978 edition. These'combined 
GR's were computed by adding theJnumber^of male nouns and pronouns' and the number of 
female' nouns and pronouns acrois the sets of Instructional Tests. The Instruction^ 
' Tests do not show a trend similar to the Survey Tests of increased GR's at the 
'upper level. This supports the. hypothesis that the increased GR's -of the Survey 
tests were a result of the Science and Social Studies items, since these two 
fcett? appear only in the Survey. 

O nqto the 1970 Advanced level. 
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TABLE -3' 



"GendeV Ratios (GR) by Level of the 1978 MAT 
Combined Across the Three Sets of Instructional Tests 



Level 

Primer 
Primary 1 
Primary 2 ■ ■ « 
Elementary 
Intermediate. 
Advanced 1 
Total i - . 



MN + MP 
FN + FP 

71/70 
126/116 
131/135 
148/138 
109/109 
123/77- 
708/645 



Combined 
v GR 

1.01 
1.09 v 
.97 . 
1.07 
T.00 
• LIP. 
1 .10 



Since the tl 978 MAT^uthors attempted' to neutralize the language within the test, a 
neuter/-ratio (NR) of neuter nouns and pronouns to gender-specific nouns and pronouns 
was computed. These data are summarized in Table 4. The NR's ranged from .16 to 
1.93 and tended to increase at the upper levels. The NR's were generally 
lowest in the Instructional Mathematics a.t the Primer through Elementary levels. 
At tf/ose lei/els', (there is an abundance of items of the type: "Sally had 
3 apples, sh'e ate' 1 . How many does she have left?" While it is possible to 
rephrase tHose items using a neuter pronoun such as "I" or "we^ in place otN 
"Sally", to do so with all items migfft make the test monotonous. It shoul^ 
also be noted that n^ter terms, especi ally-neuter job titles and generic nouns 
are usually of g-rscrter Vexing difficulty , (Beck, 1975). ^ince test item 
vocabulary is generally kept at or below grade-level, the rephrasing of -tt5mr""*~- 
into neuter forms is restated in the lower levels of the test. 



TABLE 4 



Neuter Ratio (NR): 



Neuter Nouns & Pronouns td Gender-Specific Nouns and 
Pronouns by Component and Level .' 



(MN 



NN 
MPT 



+ NP 



+ (FN + V FP) 



Instructional Tests 

Mathematics 

Primer 
\Primary 1 ' 

Primary 2 

Elementary 

Intermediate 

Advanced 1 

Reading 

Primer - 

Primary 1 

Primary 2 

Elementary 
- 'intermediate 

Advanced 1 

Language 
Primer 
Primary V 
Primary 2 
Elementary 
Intermediate 
Advanced 1 



6/29 
-18/38T 
17/45 
10/46 
30/24 
29/18 



27/44 
55/115 

£7/95 

102/91 

128/88 * 
76/100 



29/68 
70/89 
87/126 
108/149 
89/106 
.85/82 



■7 



NR 



.21 

.47 • 
.38 • 

.2ir 

.61 



.61 

.4 

l!l2 
1.45 
.76 



.43 
.79 
.69 
.72 
.84 
.04 



Median NR 
for Component 



.43 



.73 



.76 



Survey Tests 
Preprimer 
Primer 
» Primary 1 
Primary 2 
Elementary 
Intermediate , 
Advanced 1 
" Advanced 2 

Total Mathematics 
Tota-1 Reading 
To^al Language 
Total Survey 
Components"N£ombi ned 



9/55 

31/62 
107/135 
102M42 
143/137 
185/96 
130/T15 

89/70 

110/200 
. 455/533 
468/£20 
796/812 
1829/21*5 



.16* 
.50 / 

.72 

1.04 
'1.93 

1.13 
•1.27 

.55 
":85 
.75 
-.98 
.'84 



;92 
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Table 5' lists each level across all components, with the thematic breakdown 
within each level. Traditional occupations^ activities and emotions outnumber » 
the^nontraditional at almost every 1-eiiel for males and females alike. However, 

is clear that Males and males are also shown in non-stereotyped occupations 
and activities. The data indicate that more effort was made" to balance the image 
of women than theMnfage of men. For example, the raVio of females in traditional 
occupations" ^ those in non-traditional -ones is^26/28. The corresponding ratio 
for men is 90 to\. "The same pattern continues, though to a lesser degree, for 
the category of activi ties . 



An encouraging result is that the ratio of female adive' roles to passive roles 
and the ratio of mal| active to passive role^are aJ-most equal (2.53 and 2.41, 
respectively). Certainly not all females >ar e portrayed as passive and dependent, 
nor are all males seen as strong and active. In fact, females ^and males are 
both shown inactive roles more than twice as often as they are shown in 
passive roles. 

\ 

— . \j 

The balance is somewhat reversed in the category of emotions . The majority of 
males and females display sf^stereotyped emotions. However, the range of 
emotions expressed by. males was broader than that of women - which is in 
itself non-stereotypic. ' J 



J 



10 



\ 



s 



TABLE 5 



Breakdowns: by Sex and Themat\c Category of Traditional and Non-Traditional 

Situations by Level Across Components 



Occupations, ' 
Female Male 
Trad Non T Trad Non T 



"Primer 


3 


1 


2 


1 


Primary *1 


5 


: 1 


13 


0 


Primary 2 ■ 


7 


1 


18 


3 


Elementary 


.3 


3- 


16 


0 


I n termed i a£» 
Advanced "l 


5 


9 


22 




3 


13 


19 


2 


Totals 


26 


28 


90 


7 



■ Activities 
Female 



Male 



Trad Non T Trad Non T 

2 
9 

13 
20 
2 

0 . 



17 


11 


21 


22 


15 


35 


32 


20 


, 29 


33 


20 


39 


18 


10, 


21 


10 


. 13 


15 


132 


89 


160 



46 



Roles 



Emotions 



Female 



Level 

Primer 3 

Primary 1 6 

Primary 2 10 

Elementary 5 

Intermediate 6 

Advanced 1 8_ 

• Totals 38 



Active Passive 



■ Male 
Active Passi ve 



Female 



Male 



Trad Non T Trad Non T 



1 

2 
7 
3 
0 
2 



5 
6 
9 
9 

2 

10 



5 
6 
4 
1 
1 
0 



0 
4 
2 
2 
1 
4 



15 



13 



0 
0 
0 
2 
2 

_0_ 

4 



•2 
0 
7 
1 
4 
0 



14 



0 
0. 
2 
0 
1 

_5_ 
8 



41 17 

The thematic categories are somewhat difficult- to interpret. ETS (1976) - 
rlcommends that 25% or less of the items should be classifiable as "Traditional 
' Nevertheless, the issue can be framed in terms of whether the test ought to 
reflect 'society as it is_ (still with an overwhelming proportion of traditional 
models-) orNj/hether the test ought to show a more even distribution -of non-tradi 
tional and tra'ditygnal themes for women and men. , 

\ The gender balance within illustrations is summarized in Table 6. The ratios 
were .computed for each level across components, and for each component across 
levels. With the exception of the Mathematics Instructional Tests, which con- 
tained very few pictures of people, the ratios of males to female's are all t . 
very close to 1.00. This indicates an almost even split between female and 
male figures for the test as a whole and within each separate part. 
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TABLE 6 

a 

Ratio of Male to Female Figures in Illustrations 
'by Component a'nd Level 



Component (Across Levels) 

Instructional Mathematics 
Instructional Reading 
Instructional Language, 
Survey 
Total 

Level (Across Components)*. 
Primer 
Primary 1 
^Primary 2 
" Elementary 
Total 



M/F 

' 3/8 
44/44 
63/54 
169/140 
279/246 



76/77 
118/107. 
55/40 
19/14 
258/238 



Ratio 

.38 
1 .00 
107 
1.21 
1.13 



-.99 
1.10 
1 .38 
1 .36 
1.13 



The qualitative categories were rather inappropriate when applied to the 
• drawings. Most drawings were not complete enough -to permit an analysis of 
the figures' occupation, role, or emotion. 

•The ratios of number of females dressed traditionally, to the number of females 
dressed non-traditionally ranged from .50 to 3,5. Most ratios (10 out of H) . . 
were above 1.0. This could be accounted for by the fact that in today'^ society, 
dress is one of the simplest ways'to discriminate between drawings of little 
girls and little boys. 

The illustrations were coded with respect to the relative physical stature of 
males and females. ' Of the 45 drawings coded, 36-were of 4nldren and 9 were 
of adults. Table 7 provides a breakdown of these cases. 



• *The 'Intermediate', Advanced 1, and Advanced 2 levels contain no illustrations of 
q people. . / 
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TABLE 7 



.Relative Physical 'stature of Male and Female Illustrations: 
Number of Ch^jdren and Adults 



v Male taller than Female 
Female taller than Male 
Female same as Male 




Total 
13 " 
3 

29 



In the majority, of the illustrations, males and females of the same age were 
drawn as being of equal height (29 out of 45 cases.) In 13 out of 45 cases the 
males were taller. Among adults, such a portrayal may be true to life. However, 
9 out of the 13 occurred in pictures of children. The major criticism in. this 
area has been that boys are often shown taller than girls when, on the average, 
children of the' same age are very similar in height regardless of sex. In 
only 3 cases (all among children) were females depicted as being taller than 
"males. 

The '1978 MAT authors and editors made an attempt to portray boys and girls 

♦ 

and men and women in mixed-sex activities. Across-all levels and components, 
there were 63 mixed-sex groups and 20 same-sex groups. Of the same-sex 
groups, 10 were composed *of all females and TO were all male. The attempt 

9 

.to illustrate mixed-sex aroup activities appears to have been successful. 

Conclusions 

It has often been assumed that biased item content affects pupil test performance. 
Whether or not this is true, and if true, exactly how content affects performance, 
are issues which ft^fe not been dealt with adequately in research literature to 
date. Without such relevant information, test publishers and critics have been 
relying on the concept of "balance" to minimize the possibility of bias. 
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There are also no data with respect to what an "appropriate" or ideal gender- 
balance would be and no objective way of determining how an achievement series 
ought to look. Test developers have assumed than an even balance would be optimal 
in reducing content bias. 

Therefore, ^educing the sex imbalance found in earlier versions of the MAT was 
one consideration in the construction of the 1978 MAT. ^The present analysis- 
indicates there is a more even balance of male and female references, and 
generally, a less stereotyped portrayal of both sexes in th.e new version of 
the test. 

The test is not evenly nor uniformly gender-balanced acros^ all levels and 
components. However, given the problems of maintatis^g the appropriate grade- 
level vocabulary and the sex-bias/sex-stereotyping that exist in the current 
school curriculum, the 1978 $AT is certainly a significant step in the right 
direction. , 
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Appendix A 

Categories for Analysis of Gender-Balance 

Written Materials 

1. Occupations 

Female Traditional: Nurse, Teacher, Librarian, Secretary, etc. 
Male Traditional: Laborer, Professional, Principal, Boss, etc. m 
• Female Non-traditional: Professional, Laborer, Boss, Principal, etc. 
Male Non-traditional: Nurse, Teacher, Secretary, etc. 

2. Activities \^ 

Female Traditional: School,' playing with dolls, onlookers, domestic chores 

Male Traditional: School sports, games, other physical activities, 

adventurer, etc. 

Female Non-Traditional: Sports, games, ohysical activity 

Male Non-Traditional: Domestic chores, child re.aring etc. 

3. Roles 

Active: Main character, problem solver, giving help/gift 

Passive: Secondary character, needing help, recipient of help/gift 

4\ Emotions j * , 

Female Traditional: Fear, nurturance/tenderness , dependency, etc. 

Male Traditional: Aggression, courage, emotional strength, "strong 

silent type" 

Female, Male Non-Traditional: Cross-sex stereotypes 
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Appendix A (Continued) ,< 
Illustrations 

1-4. Same as for Written Materials 

5. Frequency of Hale-Female Characters t ^ 

6. Clothing * 

Female Traditional - dresses, hair ribbons, aprons, jewelry, etc. * 
Male Traditional - suits, trousers, uniforms etc. 

Female Non-Tradi tonal - uniforms (except Nurses), pants, overall a, etc. 
Male Non-Traditional - Nurse's uniform, apron, etc. 

Male/ Female Same 
Male. Larger 
Female Larger 
8. Grouping 

Mixed Sex Group 
Single Sex Grou^ 



within categories of adult or child 



